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ABSTRACT 

This paper describes the use of an automated 
performance support system, the Mental Retardation-Expert (MR-E) , in 
the training of 16 student clinicians to think "functionally" about 
undesirable behaviors of persons with mental retardation or 
developmental disabilities (MR/DD) . The MR-E models human expertise 
in behavioral diagnosis and treatment, expanded and supported by the 
current scientific literature. The MR-E assists in the development of 
functionally based treatment protocols for individuals with MR/DD who 
engage in significant disruptive behaviors. The MR-E also provides an 
interactive process of case consultation and a basic understanding of 
the relationship between functional hypotheses and treatment. A 
pretest versus posttest strategy was applied to the environmental 
group (N-16) and control group. (N-7) , producing results that 
indicated that both training and interactive experience with the MR-E 
lead inexperienced clinicians to accept its recommendations. (DB) 
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The Impact of Automated Expert Assistance on the 
Training of Mental Retardation Professionals 
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Abstract 

This presentation describes the use of an automated performance support 
system, the Mental Retardation-Expert (MR-E) in the training of students to 
think "functionally" about undesirable behaviors. MR-E models human 
expertise in behavioral diagnosis and treatment, expanded and supported by 
the current scientific literature. Typically, MR-E assists in the development 
of functionally based treatment protocols for individuals with MR/DD who 
engage in significant disruptive behaviors. However, a typical consultation 
provides not only the specific treatment information contained in the 
consultation report but also an interactive process of case consultation and a 
basic understanding of the relationship between functional hypotheses and 
treatment. The results indicated that training and interactive experience with 
MR-E lead inexperienced clinicians to accept its recommendations. 



Increasingly, computers support a variety of tasks in the field of mental retardation or 
developmental disabilities (MR/DD). Automated systems are used to manage programs (e.g., 
scheduling, financial tracking, personnel tracking, etc.), to suppon clinical services (e.g., client 
behavioral data collection and analysis, documentation of critical events, etc.), and to coordinate 
services (e.g., provide data electronically to remote sites, coordinating treatments from various 
vendors, etc.) (Jacobson, 1989). However, to date, little attention has been focused on the use of 
computers to support functional assessments or treatment planning. 
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The Mental Retardation-Expert (MR-E) is an automated performance support system that 
helps clinicians develop treatment plans for individuals with MR/DD who engage in disruptive 
behaviors. Using expert system concepts (Walls, 1989), MR-E models human expertise in 
behavioral diagnosis and treatment, expanded and supported by the current scientific literature. 
MR-E guides the clinician in conceptualizing functional hypotheses, in assessing individual client 
capabilities, and in developing specific treatment recomm'^ndations (Hile & Desrochers, in press). 
This study examines the relative contributions of some of MR-E's components. Specifically, does 
the process of learning and using MR-E increase the impact of the system's expert consultation 
reports? 

Participants and Methods 

Undergraduate and graduate students, enrolled in psychology and education classes, were 
randomly divided into experimental and control groups. The experimental design was a 2 
(experimental vs. control group) by 2 (pre-test vs. post-test) design with the groups receiving 
different levels of experience with MR-E (see Figure 1). During the pre-test, each student 
individually read and completed a protocol consisting of a standardized target client description, a 
free form data sheet to record their functional hypotheses and treatment recommendations, a 
checklist to identify functional hypotheses, and a checklist to identify treatment selections. For the 
experimental group, MR-E was demonstrated and a sample consultation was conducted to teach the 
participants about the system, its development, and its utility. Following this training, the 
experimental group, led by the first author, used MR-E to obtain consultation on the target client. 
This consultation was directed so that the results agreed with the MR-E gold standard treatment 
report for the target client. During the post-test, both tlie experimental and control groups used the 
consultation report obtained by the experimental group to help complete the same questionnaires 
used in the pre-test. The control group received the system demonstration and training after 
completing the post-test. This design allowed us to evaluate the effects of the independent variable, 
the degree of training and experience with MR-E, on clinician agreement with the MR-E gold 
standard report. 
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Figure 1. Experimental design 
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Results and Discussion 

Sixteen students participated in the experimental condition and 7 in the control group. For 
the purposes of this study, only the data from the checklist of functional hypotheses was analyzed. 
Since the consultations used by both the experimental and control groups were the same as the gold 
standard treatment report/ the students' pre-test and post-test functional hypotheses were compared 
to the MR-E standard using the interobserver reliability coefficient (lOR, Kazdin, 1982). Figure 2 
shows that the experimental group's agreement with the MR-E standard increased significantly 
following training (t (15) == 5. 11, < .001) while the control group showed no significant change. 
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Figure 2. Effect of MR-E training on sttident performance. 

These results support the importance of the interactive component in decision support 
systems. Only when clinicians interact with system, do they heed its advice. Inexperienced 
clinicians change their hypotheses only after they have come to understand MR-E and have used 
the system to obtain a consultation. The increase in agreement shown by the experimental group 
was due to their experience with MR-E. This experience included a systematic introduction to the 
concepts embodied in a performance support system, a demonstration of MR-E's various 
components, a demonstration of a functional consultation, and a group interaction with the system 
to obtain a consultation on the target case. Furthermore, the post-test level of agreement achieved 
by this group approximated the level of agreement found between experienced clinicians and MR-E 
in other studies (Hile, Ghobary, & Campbell, 1993). This suggests that MR-E may be effective in 
increasing the performance of novice clinicians to levels approximating those of their more 
experienced counterparts. 

It is also important to note that the provision of the consultation report alone did not 
increase the control group's level of agreement with the gold standard report. This finding 
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counters the fear that individuals will uncritically follow the recommendations offered by an 
automated system. In this study, inexperienced users, who might be expected to be easily 
persuaded by a report from an automated system, were not swayed by simply reading treatment 
recommendations. To have in impact the users may need to be persuaded that the system has a 
broad basis and expertise. 

Our results suggest that users are not passive recipients of expertise. As in human based 
consultations, clinicians need to accept in the qualifications of the expert, they need to interact with 
the expert to arrive at an understanding of the problem and treatments, and they need to use that 
information to reach their own conclusions. When such information is provided, MR-E enhances 
the performance of novice clinicians. 
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